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ABSTRACT 

The passage of the Kentucky Education Reform Act (KERA) in 
the 1990s made Kentucky one of the first states to initiate comprehensive 
education reform. It established a legislatively mandated goal that all 
public schools reach "proficiency” (100 of 140 points) on an accountability 
index by 2014. This paper tries to predict which schools in the state will 
reach proficiency. In 2001, with 9 years of accountability concluded, 162 
schools had garnered at least 80 points, 543 had at least 70, and 1,009 
schools had at least 60 points. Although a simple summation of yearly gains 
indicates that most schools will meet the 100-point goal, major changes in 
the accountability system make it difficult to estimate how many schools will 
make it. The report looks at three different ways to predict school 
achievement: method A accepts the past and assumes the future will be a 
linear extension; method B assumes a huge increase evident from 1998 to 1999 
was an aberration and excludes that year in making projections; and method C 
uses a 7-year average and adds this to a base score. Relying on method C as 
the more reliable predictor, the report concludes that, unless the 
accountability system is changed once again, most schools will not reach the 
100-point minimum by 2014. (Contains 10 references and 3 appendices that 
contain forecasting scores for successful schools.) (RJM) 
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THE KERA ENDGAME: 

WHICH KENTUCKY SCHOOLS WILL ACHIEVE PROFICIENCY BY 2014? 



Kentucky, one of the first states to initiate comprehensive education reform in the 
1990s, has a legislatively mandated goal that all public schools reach 100 of 140 points or 
“proficiency” on the accountability index by 2014. Several methods for projecting 
accountability scores in 2014 are compared for all Kentucky schools and for urban 
schools in the two largest school districts in the state (Jefferson and Fayette). Although 
projecting school performance so far into the future increases the likelihood of substantial 
estimation errors, the most realistic method projects that somewhat less than half of all 
Kentucky schools and only one-third of urban schools will likely reach the minimum goal 
of 100 points by the 2014 deadline. This preferred method projects that statewide no 
more than about 500 schools, and in Fayette and Jefferson Counties no more about 60 
schools will achieve the 100 point minimum, unless prior to 2014, the General Assembly 
and the KDE make another major accountability system change comparable to the change 
from KIRIS to CATS in 1999. The 269 schools across the state that appear almost certain 
to achieve proficiency by 2014 (they are projected to reach 100 or more points using all 
three methods) are listed in Appendix A. The 59 urban schools that are projected to 
achieve the minimum score using the preferred method are listed in Appendix B. 
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EDUCATION ACCOUNTABILITY ESI THE STATES 

Although all American states have joined the growing movement to hold public 
schools more responsible for their performance, they vary considerably in then- 
approaches to school accountability. Advocates of American federalism contend that this 
is the genius of the system — major complexities and unknowns in the policy area of 
education as well as diversity in state and local social and economic conditions require 
experimentation and varied approaches to policy problems. Critics of this highly 
decentralized system argue that substantial resource inequality across jurisdictions as well 
as variations in performance require greater control by the national government 
Centralized control is necessary to establish consistent, high standards and to provide 
increased and more equitable resources necessary to achieve a truly internationally 
competitive system of public education in this country. These two perspectives on 
federalism and public education are contested continually in the legislative and judicial 
arenas. 

The diversity of American public education and systems of accountability is 
documented by Education Week in their yearly evaluation titled “Quality Counts 2001 : A 
Better Balance.” As evidence of standardization and uniformity, the study finds that 49 
states have developed statewide academic standards in at least some subjects, and 50 
states test student learning in some way. As evidence of diversity and inconsistency, only 
27 states hold schools responsible for results either by rating the performance of all 
schools or by identifying low performing schools, http://www.edweek.org/sreports/qc01/ 
Of these 27 states holding schools accountable for performance, 1 1 base their ratings 
entirely on test scores. The remaining 16 states include other factors such as school 
attendance and dropout rates, but in most cases these non- test factors are not heavily 
weighted components of the overall ratings. Also, of these 27 states that hold schools 
accountable for performance, 18 evaluate school improvement over time and 19 require 
schools to meet an absolute standard, and some do both. In addition to the Education 
Week report, examination of state education department websites confirms the varied 
approaches to school accountability taken by the states. 

Components of this Education Week index or grade for accountability and 
standards include whether the state has adopted standards in core academic subjects, and 
whether its rates schools, and provides rewards, sanctions, and assistance. Of the top 
states in the Education Week assessment of standards and accountability, Maryland, New 
York, Kentucky, and New Mexico received grades of A or A-, while Massachusetts, 
North Carolina, and South Carolina received grades of B+. Six states (Tennessee, 
Minnesota, Rhode Island, North Dakota, Montana, and Iowa) received failing grades of 
F. The websites of state education agencies also provide information that helps document 
the wide variety of accountability systems. As one of the top school accountability 
systems in the study, North Carolina’s ABCs system sets growth and performance 
standards for elementary, middle, and high schools using end-of-grade and end-of-course 
tests in a variety of subjects. The end-of-grade tests are curriculum-based multiple- choice 
standardized achievement tests based on North Carolina standards aligned with national 
curriculum standards. Schools that reach specified levels of growth and have certain 
numbers of students performing at or above grade level are eligible for incentive awards. 
There are several categories of performance ranging from Schools of Excellence through 
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Low-Performing Schools, but approximately 70% of schools met either expected or 
exemplary growth standards in 2000. 

Another top-rated system. New Mexico evaluates schools and districts on five 
indicators including student achievement, attendance, dropout, safety, and parental 
involvement as well as indicators of appropriate expenditure of funds, and meeting 
additional state and federal regulations. Categories of performance range from Exemplary 
through Disapproved with 84 percent of schools meeting or exceeding standards in 2001 . 
Maryland created the Maryland School Performance Program (MSPP) in 1989 with an 
accountability system that sets standards in student achievement, dropout, and attendance 
rates. The student assessment program (MSPAP), tests students annually in grades 3, 5, 
and 8 with plans to extend the system to high schools in the near future. The system uses 
gain scores to determine levels of performance and possible financial rewards with 61 
elementary and middle schools recognized for making significant improvements over two 
to three years through 2000 and receiving monetary rewards and another 293 schools 
recognized for making one-year gains but receiving no monetary awards. Even with the 
Education Week summary and evaluation, it is difficult to briefly describe and clearly 
categorize the many variations in state education accountability systems that have 
developed over the past decade or more. The next section describes the Kentucky 
accountability system that shares some similarities to other state systems, but also 
exhibits certain unique features. 

KERA’S OPERATIONAL GOAL: 100 POINTS BY 2014 

Kentucky is one of the few state systems that not only rates schools based on 
performance over time, but also has an ultimate, operational goal for school performance. 
The goal for all schools is a clear standard that must be achieved within a designated time 
limit in order for the school to be judged successful or proficient Kentucky, one of the 
first states to initiate a comprehensive education reform program in the 1990s, has a 
legislatively mandated goal that all public schools reach 100 of 140 points on the 
accountability index by the year 2014. A score of 100 points is considered to be 
proficient, one of the four performance levels on the scale ranging from novice through 
apprentice, proficient, and distinguished. Not only are all schools expected to reach 100 
points by 2014, they are evaluated and classified every two years based on their progress 
towards this minimum goal. 

Each Kentucky school now has a growth chart that uses a straight line to project 
from their baseline score in 2000 to the goal of 100 points in 2014. 
http://apps.kde.state.ky.us/cats reports/ Schools that score above the goal line at each 
two-year interval are considered to be meeting their goal and receive monetary rewards. 
Schools that score below the goal line but are above what is called the assistance line are 
considered to be progressing and are eligible for a lesser level of reward as long as they 
score above their previous baseline score. Finally, schools that score below the assistance 
line receive no rewards and the lowest one-third of these schools must undergo a 
scholastic audit to determine how much and what type of assistance is needed. The 
remaining schools below the assistance line undergo other types of reviews. In addition to 
accountability scores, in order to receive rewards schools must meet certain other targets 
for reducing their dropout rates and their proportion of students classified as novice. 
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Before considering alternative forecasts of the performance of Kentucky schools 
in the ultimate accountability year of 2014, it is important to understand some of the 
difficulties in projecting organization behavior, especially far into the future. Regardless 
of the subject, forecasting is always risky business, especially when attempting to project 
behaviors more than a few years or even a few months into the future. Weather 
forecasters develop complex models with extensive data designed to analyze many 
interrelated variables as they attempt to predict weather accurately days or even hours in 
advance, let alone weeks or months in advance. Economists also have developed complex 
models to predict aggregate economic behavior, often with tittle success, especially when 
projecting far into the future. Experts in time series analysis caution that projections more 
than a few data points into the future are likely to be characterized by large estimation 
errors making them very unreliable predictions of future behavior, especially when 
unexpected events occur outside the boundaries of the model. A recent example is the 
September 1 1 terrorist attack on the U.S. and its substantial but largely unknown effects 
on the nation’s economy and other social systems and political institutions. As will be 
shown below, despite tire popular image of relatively stable and conservative 
organizations reinforced by the reality of considerable stability in the actual performance 
of Kentucky’s schools, predicting the behavior of these coping organizations (Wilson, 
1989) is neither simple nor certain, especially beyond a one or two-year period. 

ASSUMPTIONS ABOUT SCHOOL PERFORMANCE 

This analysis of Kentucky school accountability begins with three assumptions. 
First, it is assumed that the performance of individual schools does not change 
substantially from one year to the next - - most yearly changes in accountability scores 
fall within a limited range. Second, aggregate school performance is moderately to highly 
correlated from one year to the next — the rank order of school performance is relatively 
stable from year to year; and third, although subject to error (random and systematic), 
reasonably accurate projections of future school performance can be made based on past 
performance. The third assumption regarding projections is the primary subject of this 
paper and will be examined in some detail, however projections of future behavior 
depend to a great extent on the reasonableness or validity of the first two assumptions. 

Table 1 describes accountability change scores from 1993 through 2001 and with 
one exception provides much support for the first assumption. Except for the 
extraordinarily large gain in scores from 1998 to 1999, most yearly changes in 
accountability scores fall within a limited range and are distributed relatively consistently 
over time. There are few exceptionally high increases or decreases in scores from year to 
year. The unusually large increase in accountability scores from 1998 to 1999 will be 
examined in more detail below. 
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TABLE 1 

CHANGES IN SCHOOL ACCOUNTABILITY SCORES 






YEA 


RLY CHANGE SCORES 




93-94 


94-95 


95-96 


96-97 


97-98 


98-99 


99-00 


00-01 


Total n 


1156 


1130 


1180 


1161 


1173 


1169 


1186 


1192 


Mean ch 


5.7 


4.0 


-1.4 


4.3 


-0.5 


16.8 


1.9 


2.5 


S.D. 


6.4 


6.0 


5.6 


4.9 


4.3 


5.7 


4.0 


4.2 


Min 


-15.7 


-19.5 


-31.0 


-16.2 


-28.8 


- 10.0 


-16.3 


-15.2 


Max 


51.4 


33.9 


26.5 


21.7 


17.6 


38.5 


21.4 


25.9 


>0 


970 


856 


428 


959 


506 


1166 


819 


872 


>5 


584 


469 


127 


499 


96 


1142 


212 


270 


>10 


237 


148 


26 


126 


12 


1031 


35 


52 


>15 


86 


41 


10 


17 


2 


733 


4 


10 


<0 


178 


270 


746 


197 


651 


3 


358 


308 


>-5 


25 


53 


269 


41 


135 


1 


39 


31 


>-10 


8 


11 


46 


9 


19 


0 


4 


2 



Table 2 shows that after the earlier years of KERA when one might expect much 
diversity and unpredictability as schools struggle to adjust to a new system of 
accountability, aggregate school performance is related closely from year to year. 
Beginning in about 1996, the rank order of school performance begins to display 
moderate to strong stability culminating in the very high correlations for school 
performance in 1999 and 2000 (r = .92) and in 2000 and 2001 (r = .91). These strong 
correlations show that in the late 1990s and in the early 2000s, schools have settled into a 
relatively stable pattern of performance - higher scoring schools continue to score higher 
while lower scoring schools continue to score lower. Very few schools are now making 
comparatively major gains or declines in performance such that they would leap ahead or 
fall behind many other schools and weaken the correlations in scores from one year to the 
next 



TABLE 2 

SCHOOL ACCOUNTABILITY SCORE CORRELATIONS * 





1993 


1994 


1995 


1996 


1997 


1998 


1999 


2000 


1993 


















1994 


.56 
















1995 


.54 


.68 














1996 


.54 


.62 


.71 












1997 


.58 


.59 


.62 


.75 










1998 


.52 


.58 


.58 


.69 


.84 








1999 


.60 


.56 


.61 


.69 


.75 


.82 






2000 


.58 


.52 


.57 


.67 


.73 


.78 


.92 




2001 


.54 


.50 


.56 


.67 


.72 


.75 


.85 


.91 



* N’s range from 1047 to 1 192 
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SCHOOL PERFORMANCE 

Before considering alternative forecasting methods to project future school 
performance, the current status of Kentucky schools relative to the goal of 100 points and 
proficiency is examined. What are the scores of the approximately 1200 schools in 2001 
as they face 13 more years of work to reach the goal of 1 00 points? As of 2001, with nine 
years of accountability concluded (eight yearly changes) and with thirteen years 
remaining to the deadline in 2014, 162 schools have reached at least 80 points, 543 
schools have reached at least 70, and 1 009 schools have reached at least 60 points. In 
addition, four schools (two from one small school district) have already reached the 
minimum of 100 points and thirty schools have reached 90 points and higher. Although 
these scores in 2001 look very promising for future success, how might projections of 
future performance be made? 

To make an initial but somewhat misleading projection based on a simple 
assumption, suppose Kentucky schools on average gain 2.5 points each year from 2001 to 
2014. If this occurs, one could expect that on average most of the 543 schools that have 
reached 70 points by 2001 would attain the necessary minimum by 2014 (13 x 2.5 = 32.5 
+ 70 = 102.5 points). Recognizing that this assumption might bring somewhat less than 
half of Kentucky’s schools to the minimum standard, is 2.5 points per year of average 
gain a reasonable assumption? 

Since the average yearly change from 1993 to 2001 for all schools that existed 
throughout the period is 4.1 points (33 points divided by 8 yearly changes), the 
assumption of 2.5 points per year seems very conservative, suggesting that a substantial 
majority of schools should reach the minimum score by the deadline. If one takes the 
average yearly change from 1993 to 2001 for each school and multiplies this figure by 13 
(years to 2014), then adds this number to each school’s score in 2001 (the current base 
year), 917 of 1078 schools that existed in 1993 and have continued through 2001 are 
projected to reach or surpass 100 points by 2014. [Schools that have closed, merged, or 
opened since 1993 are not included in this calculatioa] This is probably the simplest 
projection in that it takes school performance over the past 9 years and projects that 
performance 13 years into the future. This method also results in a high level of success 
in that approximately 8 of 10 schools are projected to reach the minimum goal of 
proficiency by the deadline year. 

Before accepting the assumptions and calculations of this simple method with its 
projection of a relatively high proportion of successful schools, the data in Table 3 raise 
several questions. The table shows that the average gain of 4.1 points per year from 1993 
to 2001 masks a wide range of variation from year to year. For example, scores increased 
an average of 1 1.7 points from 1998 to 1999 (before adjustments to 1999 scores made in 
2001), but decreased 1.4 points from 1995 to 1996 and a half point from 1997 to 1998, 
indicating that the pattern from 1993 to 2001 is not regular, incremental increases over 
time. This apparent lack of trend combined with the unusually large gains made in 1999 
suggests alternative ways to calculate and compare projections based on school 
performance over the nine years of testing. 
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TABLE 3 

KERA ACCOUNTABILITY SCORES OVER TIME 




YEAR 


N 


MEAN 


MIN-MAX 


AVGCH/YR 


1993 


1158 


35.9 


19.1 - 69.8 












5.70 


1994 


1186 


41.5 


24.6 - 76.5 












4.02 


1995 


1180 


45.6 


25.1 - 79.6 












-1.43 


1996 


1181 


44.2 


24.2 - 75.6 












4.30 


1997 


1145 


48.4 


27.6 - 80.9 












-0.48 


1998 


1179 


48.0 


25.1 - 77.4 












11.72 


1999’" 


1207 


59.7 


33.3 - 88.7 




1999 (adj)** 


1184 


64.9 


35.9 - 99.2 


16.84 (adj) 


2000 


1217 


61.1 


35.6-94.5 




2000 (adj)’"’" 


1192 


66.7 


36.9- 103.2 


1.86 (adj) 












2001 


1210 


69.1 


39.7- 106.9 


2.46 (adj) 




* In 1999, the accountability system changed from KIRIS to CATS. 

** In 2001 the accountability scores for 1999 and 2000 were adjusted to reflect changes in 
standards 



At least three tilings stand out in comparing the changes in accountability scores 
from 1993 to 2001 presented in Table 3. First, as might be expected, schools on average 
made substantial gains in the first two years of accountability — a 16 percent increase (5.7 
points) froml993 to 1994 and a 9.6 percent increase (4.0 points) from 1994 to 1995. 
There are several possible explanations for this initial strong performance, but two that 
seem most likely are (1) testing experience or learning - schools (students and teachers) 
learned much about the accountability tests over the first two years of experience (the 
learning curve was steep initially), and/or (2) some schools in 1992/1993 may not have 
tried to establish a strong initial score knowing that this score would serve as the baseline 
for measuring change over time and these gains or declines would lead to future rewards 
or penalties. 

The next noteworthy characteristic of these data is the size of the average change 
score from 1998 to 1999. The increase of 1 1.7 points or almost 25 percent in one year 
(prior to an adjustment made in 2001) is more than double the next highest increase that 
occurred in the first year from 1993 to 1994. The third characteristic of the data, also 
related closely to the large gain in scores in the 1998/99 academic year is the adjustment 
of 1999 and 2000 scores made in 2001 . With these latest adjustments, the mean score in 
1999 increases from 59.7 to 64.9 points and the mean score in 2000 from 61.1 to 66.7 
points. The change from 1998 to the adjusted 1999 scores has now become a substantial 
16.8 point increase or an average gain of 35 percent in one year. The change from 1998 
to the adjusted scores in 2000 has now become an increase of 39 percent or an average 
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gain of about 20 percent each year from 1998 through 2000. There may be other plausible 
explanations for these dramatic increases in 1999 and 2000 scores, but the major change 
in the accountability system from KIRIS to CATS implemented by the KDE in 1999 and 
the additional scoring adjustments made in 2001 stand out as the most likely reasons for 
the large increases. 

WHAT HAPPENED TO ACCOUNTABILITY IN 1999 AND 2001? 

The short answer to the question of what happened in 1998/99 is that the 
Kentucky General Assembly and the Department of Education changed the 
accountability system from KIRIS to CATS. Appendix C includes a summary timeline 
for the KERA accountability system in the period 1997 through 2001 that hi ghlig hts 
some of the changes made in the transition from the original KIRIS testing system to the 
CATS system initiated in 1999. 

After about 7 years of KERA and 5 years of accountability and testing, policy 
makers faced several highly contentious issues. These included whether a national, 
standardized or norm-referenced test should be used along with or instead of the custom- 
made tests based on specific content and standards developed by state educators, the use 
of rewards as salary bonuses to staff, the types of sanctions imposed on poorly 
performing schools, and whether each school should have a fixed improvement goal. A 
Task Force examined these and other issues and issued a 1998 report that recommended 
using a national standardized achievement test, increasing the emphasis on basic 
academic skills, simplifying school achievement goals, prohibiting cash rewards used as 
bonuses for teachers, and developing a school report card (Harp, 1997). 

At about the same time, a report from a group of national testing experts 
commissioned by the Legislative Research Commission criticized the KIRIS 
accountability system and recommended a new process be initiated to reestablish a 
baseline year and reset accountability standards as soon as possible (Blackford, 1998). 
Towards the end of the 1998 legislative session, the General Assembly approved HB 53 
requiring that a new Commonwealth Accountability Testing System (CATS) replace the 
previous KIRIS system and include a norm- referenced test, open- response and multiple- 
choice items, writing portfolios but with reduced time for students and teacher 
involvement, and more participation of teachers in designing and scoring the new tests. 

The legislation also required the establishment of a school report card, reconfigured the 
consequences section of accountability, and required the State School Board to establish 
a new formula for accountability and school improvement 

A School Curriculum, Assessment, and Accountability Council was appointed to 
advise the state School Board on further development of the new state testing system 
including whether the test should be designed to provide reliable scores for individuals 
students, schools, or both; the mix of multiple-choice and essay questions; and how much 
of the test would be customized for Kentucky or consist of national standardized test 
items (Harp, 1998). In June 1998, the state school board approved an RFP for a 
contractor to build a new accountability test and in September awarded CTB/McGraw 
Hill a four- year $30 million contract to design and score CATS as the replacement for 
KIRIS accountability system. 

In late 1998, the state school board proposed a new system to evaluate and 
classify schools. The heart of the new system is a “line of expected growth” that is 
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plotted for each school showing how it must progress from 2000 to reach the goal of 100 
points by 2014. http://www.kde.state.lcv.us/comni/commrel/cats/long term.asp Every two 
years, a school will be classified as meeting its goal, progressing, or needing assistan ce. 

In early 1999, based on recommendations from The National Technical Advisory Panel 
on Assessment and Accountability and WestEd, the assessment subcontractor for the 
Kentucky Core Content Tests, that the core content be reviewed prior to the development 
of new questions, committees begin benchmarking (aligning to national standards) in five 
content areas and in April 1 999, the first CATS test was taken. After two years of testing 
and accountability using the CATS system, recalculated scores for 1999 and 2000 were 
released in August 2001 in the form of growth charts for each individual school. 

Although the major change in the accountability system occurred in 1999 when the 
system changed from KIRIS to CATS, as shown in Table 3, further adjustments made in 
2001 to 1999 and 2000 CATS scores are also substantial. The current Commissioner of 
Education contends that the “new student performance standards came from the most 
comprehensive standards- setting process ever undertaken by any state- level education 
testing system” (Wilhoit, 2001). 

This major change in accountability systems raises the question of whether it is 
reasonable to use pre- 1999 scores in longitudinal models to project future performance. 
Does the new accountability system and revised achievement test make comparisons of 
school performance before and after 1999 suspect or invalid? This is a reasonable 
question, but I note that the accountability scale remains the same throughout the period 
(0- 140 points), different groups of students and different grades have taken the tests 
throughout the entire period, and there were many adjustments in the accountability 
system prior to 1999. 1 believe it is reasonable and informative to use the earlier scores 
and although the 1999 change accomplished many important and perhaps necessary 
goals, I contend that the primary result was to raise the baseline score substantially for 
almost all schools. As the following sections show, school performance data from 1993 
through 1999 provide important and useful information that helps understand how 
schools performed in the past and possibly might perform in the future. Arguing that 
these earlier scores are not valid measures of organization performance or pretending 
they don’t exist seems unreasonable and short-sighted. 

ALTERNATIVE PROJECTIONS OF SCHOOL PERFORMANCE 

Given the unusually large average increase in scores in 1999 (including the 2001 
adjustments), an alternative to the projection method described previously is to ask what 
might the average gain be without die huge one-year increase in 1999 that accounts for 
about half of the total average increase of 33 points from 1993 to 2001? For example, if 
one takes the average yearly change for each school that occurred prior to the major 
system adjustment in 1999 (1993- 1998) and multiplies this figure by 16 years (1998 to 
2014), then adds this figure to each school’s score in 1998, only 317 schools would be 
projected to reach the goal of 100 points by 2014. This projection is labeled Method B 
and can be compared to the previous method labeled Method A in Table 4. 

Comparing Method A to Method B in Table 4 shows that the system adjustment 
in 1999 is likely to have a huge positive impact on school performance over the next 1 14 
decades. Given certain assumptions, this change in 1999 could mean the difference 
between only about one of three schools reaching the overall goal compared to about four 
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of five schools reaching the goal (917 versus 318 schools). However, even if as some 
critics suggest, the 1999 scores do not indicate real learning or true school performance 
and therefore artificially inflate average change scores from 1993 to 2001, since the large 
gain in 1 999 is unlikely to be adjusted down between now and 2014, Method B is not 
realistic and is useful only as an analytical comparison. Regardless of questions about the 
statistical, methodological, or pedagogical legitimacy of the huge overall increase in 
accountability scores in 1999 (and the additional adjustments made in 2001), there seems 
to be little or no possibility that the scores for that year will ever be adjusted downward. 
Given political and policy realities, an alternative projection method that combines 
elements of both Method A and Method B may provide a more realistic and accurate 
projection of school performance. 

The third example of a projection of school performance, labeled Method C in 
Table 4, accepts that the huge overall gain in 1999 scores is now a permanent part of the 
baseline for each school, but does not use the exceptionally large increase from 1998 to 
1999 to calculate yearly averages for projections. Even with the inclusion of the large 
average gain in 1999 and the later adjustments to 1999 and 2000 scores leading to the 
average score of 69.1 in 2001, if one assumes that the average gain per year would more 
likely be that of the other 7 year changes (omitting the large gain of 1 6.8 points from 
1998 to the adjusted 1999 scores), then the average change score in this period is 2.34 
points. When the average yearly gain for reach school using this method is multiplied by 
13 years and added to the 2001 score of each school, 483 schools are projected to reach 
the goal of 100 points by 2014. 





TABLE 4 




PROJECTIONS OF 2014 ACCOUNTABILITY SCORES * 






METHOD A 


METHOD B 


METHOD C 


Trend Years 


1993-2001 


1993-1998 


1993-1998 

+1999-2001 


Avg Yearly Change 


4.12 


2.40 


2.34 


Base Year for Projection 


2001 


1998 


2001 


Mean Score 2014 


122.5 


86.4 


99.3 


# Schools 100+ pts in 2014 


917 


318 


483 


# Schools 98+ pts in 2014 


937 


339 


513 


# Schools 140+ pts in 2014 


232 


34 


41 


# Schools < 80 pts in 2014 


16 


455 


188 




* The data are for all Kentucky schools with N’s ranging from 1047 to 1085 depending on 
projection method, and using 1999 and 2000 scores as adjusted in 2001. 



This third alternative (Method C) accepts that the system change made in 1999 
cannot be assumed away as in Method B, however it does assume that the more realistic 
future pattern of accountability scores should ignore the huge gain in scores from 1 998 to 
1999 and use a seven-year instead of eight- year average. With this method, the current 
year 2001 base is accepted (keeping the adjusted scores from 1999 through 2001) but the 
average projected gain from 2000 to 2014 is based on average change from 1993 to 1998 
and from 1999 to 2001 (seven instead of eight years). This method projects that 
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somewhat less lhan half (45 percent) of Kentucky schools are likely to reach the goal of 
100 points by 2014. It should be emphasized that these forecasts are based on schools that 
existed and continued through 2001. Although some schools in 1993 no longer exist and 
new schools have been added since 1993, there are not sufficient numbers of these 
schools to make any noticeable difference in the overall projections of successful schools 
using these three methods. However, a cursory examination suggests that the newer 
schools are performing at higher levels than most of the schools that closed or were 
merged, so the final count of successful schools would likely be somewhat higher than 
projected in Table 4. 

To summarize the three projection methods, Method A accepts the past and 
assumes the future will be a simple linear extension of that past (8 years are used to 
predict the next 13 years). Method B assumes the huge increase from 1998 to 1999 is an 
anomaly due to a major policy intervention in the accountability system and leaves out 
that year in projecting the future (5 years are used to predict the next 16 years). Method C 
leaves adjusted 1999 scores in the base total for 2001, but uses a 7 year average (omitting 
the large gain in 1999 to calculate a seven year average gain score) and adds this 
projected trend to the current 2001 base to project to 2014. Another way to think about 
Method C is that the high proportion of schools achieving the goal of 100 points and 
higher based on Method A (about 80 percent) is not likely to occur without another major 
policy intervention in the accountability system sometime before 2014, or the highly 
unlikely occurrence of an exceptionally large average real gain in scores sometime 
between 2001 and 2014. 

To provide additional information, Appendix A lists the schools across the state 
that reach the minimum score of 100 points by 2014 using all three projection methods. 
These 269 schools can be viewed as the consensus successes and are most likely to meet 
the minimum standard, however as Table 4 indicates many other schools are projected to 
reach the minimum using Method C and especially Method A. The data indicate that the 
three methods with differing assumptions can have substantial impacts on projected 
future scores. Some schools display differences of 30 to 40 points from one method to 
another in projected scores in 2014. Another point to make about the three methods is 
that the most “liberal” projection (Method A) predicts that many of these successful 
schools will score above the maximum of 140 points in 2014. Table 4 shows that 232 of 
1078 schools are projected to score above 140 points using Method A. This is further 
evidence that simply projecting the next 13 years based on the past 8 years may be 
misleading or too liberal. 

As an additional piece of information, the projected scores in 2014 using the three 
methods are moderately to closely correlated. The correlation for scores based on 
Methods A and B is .62; for Methods A and C, it is .90; and for Methods B and C, it is 
.75. Although the absolute values of projected scores for each school may vary 
substantially depending on the projection method, the relative ranking of performance for 
these schools does not change very much depending on the method used. 

ONE-PERIOD- AHEAD FORECASTS 

To provide further comparisons of these three methods that project far into the 
future, I use similar methods to project actual scores in 1999, 2000, and 2001 (one- 
period-ahead forecasts). Tables 3 and 4 above use existing data (yearly accountability 



scores from 1993 through 2001 including the adjustments made in 2001 to 1999 and 2000 
scores) to project future behavior that is unknown (accountability scores in 2014). 

Ostrom (1990: 77) uses the term ex -ante - forecasts for models that project into the 
unknown future and compares these to ex -post- forecasts that use part of an existing 
dataset to forecast other data points in that same dataset. His example uses a sample from 
an existing dataset to forecast nonsample data points, but here I will use data from 1993 
through 1998 and 1999 and 2000 to forecast actual scores from 1999 to 2001. This should 
provide some evidence for the relative accuracy of alternative ex-ante forecasting 
methods used in Table 4 and also further highlight the unusual system behavior in 1 999. 

Table 5 shows that Method J (average yearly change scores from 1993 through 
1998 used to predict scores in 1999) substantially underestimates actual scores and again 
emphasizes the unusually high average scoring increase in 1999. The mean difference 
between projected scores and actual 1999 scores is -14.2 points compared to mean 
differences ranging from - 0.1 to 3.0 points for the other methods and time periods. 

Method M is comparable to Method C presented in Table 4 in that it uses change from 
1993 to 1998 and 1999 to 2000 as the trend and 2000 as the base to predict 2001 scores, 
but does not include the huge increase in 1999 as part of the trend. 



TABLE 5 

PROJECTIONS OF 1999-2001 ACCOUNTABILITY SCORES 






Method J 

( 1999 ) 


Method K 
(2000) 


Method L 
(2001) 


Method M 
(2001) 


Trend Years 


93-98 


93-99 


93-00 


93-98+99-00 


Avg Yearly Ch 


2.40 


4.80 


4.38 


2.32 


Base Yr for Projection 


1998 


1999 


2000 


2000 


Projected Mean 


50.3 (1999) 


69.5 (2000) 


70.9 (2001) 


68.7 (2001) 


Actual Mean * 


64.7(1999) 


66.5 (2000) 


68.9(2001) 


68.9 (2001) 


Mean Difference (s.d.) 


-14.2 (6.1) 


3.0 (4.5) 


1.9 (4.5) 


- 0.1 (4.4) 


# Differences > 0 


13 


835 


753 


535 


# Differences > 5 


3 


352 


236 


95 


# Differences >10 


1 


42 


26 


10 


# Differences < 0 


1049 


230 


313 


512 


# Differences > -5 


995 


47 


70 


116 


# Differences > -10 


802 


11 


12 


21 


Correlation ** 


.79 


.91 


.91 


.91 


N = 


1062 


1066 


1067 


1047 




* Actual mean is for all schools projected from 1993 rather than for all schools reporting scores in 
each of the projected years (1999-2001) and adjusted 1999 and 2000 scores are used. 

** The correlation is between the estimated and actual scores for each year. 



When comparing Method M to Method L which includes the large 1999 increase 
as part of the trend to predict 2001 scores (and is comparable to Method A in Table 4), 
Method M has a much smaller mean difference between projected and actual 2001 scores 
(-0.1 vs. 1.9), and has errors more evenly balanced between under and over predictions. 
These differences show that Method M (comparable to Method C above in Table 2) is 
the more accurate method for predicting scores in 2001, and therefore also may be the 
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more accurate method for projecting 2014 scores. Although a simple model, it would be a 
challenge to derive projections much more accurate than those provided by Method M. 

There are other more complex and sophisticated forecasting and quasi- 
experimental methods that could be applied to these data. For example, the Engineering 
Statistics Handbook provides excellent discussions of moving average, exponential 
smoothing, and Box- Jenkins models 

http://www.itl.nist.gov/div898/handbook/pmc/section4/Dmc4.htm . 

Although various types of causal models could be applied to these data, this analysis is 
not causal in that it simply projects future behavior of schools based on past behavior 
rather than on theoretical constructs or characteristics of these schools that may determine 
performance. Previous research using these same accountability data and other school 
data in multivariate cross-sectional models finds that school poverty (proportion of 
children eligible for subsidized meals) is a substantial and consistent determinant of 
performance controlling for several other plausible determinants. 
http://www.ukv.edu/~proeder/keraweb.htm To provide a rough comparison to these 
previous findings showing the importance of poverty, using Method C (from table 4) the 
mean proportion of students eligible for subsidized meals for those schools projected to 
score over 120 points by 2014 is 45 percent compared to 60 percent for those projected to 
score less than 80 points by the deadline. 

Despite its simplicity, I contend that this analysis using Method C provides a 
reasonable way to project future accountability behavior taking into account the unusual 
circumstances occurring in 1999 and 2001, and especially the problem of attempting to 
forecast far into the future. Testing experts likely disagree over how much of these large 
increases in 1 999 (as well as all the testing using the KIRIS and CATS systems) are based 
on reliable and valid measures of student learning and therefore indicate real 
improvements in school performance. Regardless of this important question, the data 
suggest that the policy interventions made in 1999 and 200 1 will have a substantial 
positive impact on the likelihood that many schools will reach the mandated goal of 100 
points by 2014, however without another similar intervention, it is likely that no more 
than half of Kentucky schools will reach the minimum goal by 2014. 

Although all three methods depend on assumptions that may or may not be 
reasonable and leave substantial room for error, especially thirteen years into the future, 
based on the one-point -ahead projections of actual data in Table 5, 1 believe Method C 
presented in Table 4 is the more accurate and realistic way to project scores in 2014. The 
huge gain in 1999 scores appears to be based more on the policy intervention than on 
school performance and therefore likely distorts frue school performance from 1998 to 
1999. 1 have no other empirical data or methods to support my assumption that the huge 
gain in 1999 scores is more the result of a policy intervention rather than a frue 
improvement in student and school performance in that one year, however there is little 
doubt that the pattern of changes in scores from 1998 to 1999 is very different from the 
changes for every other year in the period under consideration. Consequently, the most 
likely scenario is that no more than about 500 schools are likely to meet the 2014 
standard, assuming that prior to the deadline, the General Assembly and the KDE do not 
again change the system as they did in 1999 followed by recalculations in scoring as 
occurred in 2001. 
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PROJECTING URBAN SCHOOL PERFORMANCE 

In this section, the above projection methods will be applied to urban schools in 
Kentucky, or schools located in the two largest school districts - Jefferson and Fayette 
Counties. As Table 6 indicates, the behavior of approximately 180 urban schools is 
similar to all schools in Kentucky. For example, with the 2001 scoring adjustments, the 
increase from the original 1999 scores to the adjusted 2000 scores is a substantial 5.9 
points. Also, the average increase of 1 1.2 points from 1998 to the original 1999 scores 
when the system changed from KIRIS to CATS becomes a 15.6 point increase with the 
adjusted 1999 scores. Of the total average increase of 30 points from 1993 to 2001 (35.8 
to 66.2 points), two-thirds or almost 20 points come after 1998. The first five years of 
accountability for these urban schools show an average total increase of just under 1 1 
points, while the most recent three years including the 1999 system change and the 2001 
scoring adjustments show an increase of almost 20 points. It is interesting to note that 
although almost all 1999 and 2000 scores in Jefferson and Fayette counties were adjusted 
up in 2001, twenty-six schools were adjusted down (4 schools in Fayette and 22 in 
Jefferson County). The 2001 scoring adjustments ranged from a decrease of 9.7 points to 
an increase of 23.3 points. 



TABLE 6 

KERA ACCOUNTABILITY SCORES OVER TIME FOR URBAN SCHOOLS 

(Jefferson and Fayette Counties) 




YEAR 


N 


MEAN 


AVGCH/YR 


MIN-MAX 


1993 


181 


35.8 




19.8-69.8 








5.29 




1994 


181 


41.1 




24.6-76.0 








4.11 




1995 


181 


45.2 




27.4-76.3 








-3.11 




1996 


181 


42.2 




24.2-73.1 








5.04 




1997 


181 


47.2 




27.6-80.9 








-0.74 




1998 


183 


46.6 




25.1-77.4 








11.22 




1999* 


183 


57.8 




33.3-86.1 


(1999adj)** 


182 


62.3 


15.64 (adj) 


35.9-97.4 


2000 


182 


59.0 




35.6-87.1 


(2000adj)** 


183 


63.7 


1.41 (adj) 


39.0-101.0 












2001 


183 


66.2 


2.53 (adj) 


39.7-99.8 




* In 1999, the accountability system changed from KIRIS to CATS. 

** In 2001 the accountability scores for 1999 and 2000 were adjusted to reflect changes in 
standards 



Table 7 compares projection methods used previously for all schools in Table 4 
and again shows the importance of the change from KIRIS to CATS in 1999 and the 



2001 scoring adjustments for the success of urban schools by the 2014 deadline. The 
table also shows that the projected performance of these urban schools using the three 
methods is similar to the performance of all Kentucky schools, however the average 
yearly changes and the projected scores in 2014 for the three projection methods tend to 
be somewhat lower for urban schools than for all Kentucky schools. As with the data for 
all schools in Table 4, the simple linear projection from the initial eight years to the next 
13 years (Method A) shows the highest proportion of successful schools (125 or just over 
two-thirds of the urban schools), while Method B which ignores the major system 
changes after 1998, projects the lowest proportion of successful schools in 2014 (43 or 
about one fourth of the urban schools). 

Projections for urban schools using Method C fall between the other two methods 
with 59 schools (about one-third) projected to reach the minimum of 100 points by 2014. 
Recall from Table 4 that this same method projects that about 45 percent of all Kentucky 
schools will reach the goal. Appendix B lists the urban schools that are projected to reach 
the minimum performance standard in 20 14 using Method C and shows that about 40 
percent of Fayette County schools will likely reach the goal, while approximately 30 
percent of Jefferson County schools will likely reach the goal. Each of the methods 
projects that Kentucky’s urban schools are likely to be somewhat less successful in 
achieving the minimum score by the deadline than the rest of Kentucky’s schools. 





TABLE 7 






PROJECTIONS OF 2014 ACCOUNTABILITY SCORES FOR URBAN SCHOOLS* 






METHOD A 


METHOD B 


METHOD C 


Trend Years 


1993-2001 


1993-1998 


1993-1998 

+1999-2001 


Avg Yearly Change 


3.77 


2.14 


2.09 


Base Year for Projection 


2001 


1998 


2001 | 


Mean Score 2014 


114.9 


80.6 


93.2 


# Schools 100+ pts in 2014 


125 


43 


59 


# Schools 98+ pts in 2014 


131 


46 


64 


# Schools 140+ pts in 2014 


29 


3 


3 


# Schools < 80 pts in 2014 


10 


92 


54 




* N’s range from 180 to 182 depending on projection method and using 1999 and 2000 scores as 
adjusted in 2001. 



Recall that Method C accepts that the huge overall gain after 1999 is now part of 
the baseline, but does not include the unusually large increase when calculating yearly 
averages used to project future performance. With Method C, the current year 2001 base 
is accepted (keeping the high scores from 1999 and the adjustments made in 2001 to 
1999 and 2000 scores) but the average projected gain from 2000 to 2014 is based on 
average change from 1993 to 1998 and from 1999 to 2001 (seven instead of eight years). 
Again, one of the problems with Method A is that many schools are projected to score 
well above the maximum of 140 points using the linear projection based on the eight-year 
average from 1993 to 2001. Using Method A, 29 schools are projected to score over 140 
points by 2014 with several projected to score over 1 80 points, while Method C projects 




16 



only 3 schools scoring over 140 points. The data and analysis suggest that officials in 
Fayette and Jefferson County School Districts have much work to do to bring more 
schools to proficiency by the 2014 deadline. 

THE KERA ENDGAME 

This research project began with three assumptions - - (1) the performance of 
individual schools does not change substantially from one year to the next (most yearly 
changes in accountability scores fall within a limited range), (2) aggregate school 
performance is moderately to strongly correlated from one year to the next (the rank 
order of school performance is relatively stable from year to year), and (3) although 
subject to error, a reasonably accurate projection of fiiture school performance can be 
made based on past performance. Examination of accountability data finds substantial 
support for the first two assumptions. More extensive analysis of the third assumption 
also finds it to be reasonable. Tables 4 and 7 compare methods for projecting school 
performance far into the fiiture and suggest at least one reasonable projection of future 
school performance that accounts for the unusual system behavior in 1999 and considers 
the effects on future performance. 

On the surface this analysis may seem to be a superficial exercise in that it 
projects future school performance based on a somewhat simple model, but the issue of 
how high to set the performance bar and how much time will be given to school systems 
to reach the goal has important political and policy implications. Most policy makers and 
reformers in Kentucky recognized this issue as they developed the accountability system 
beginning in the early 1990s and revised it in the late 1990s. The key accountability 
policy dilemma or trade-off relates to setting the ultimate standard or goal. Most would 
likely agree that the ultimate goal should be high enough to be defensible and challenging 
for all schools, but not so high that a large proportion of schools fail to meet the 
minimum goal. The many academic, legal, and political issues in standard- setting are 
discussed by Viadero (2001). 

The major adjustment in the Kentucky system made in 1998/99 (KIRIS to CATS) 
suggests the bar might have been set too high initially. Since this analysis of Kentucky’s 
accountability standard suggests that without another substantial change in the system, 
somewhat less than half the schools will fail to meet the minimum goal, one might 
conclude that the goal still may be too high. Over the next few years, policy makers, 
educators, citizens, and parents may have to decide if a 50 percent success rate is 
sufficient to consider KERA a successful education reform. 

If making projections of future school behavior based on relatively “hard” data is 
subject to many potential sources of error, then attempting to predict political, policy, and 
organizational futures based on these data is an even more hazardous undertaking. Also, 
as 2014 approaches, the future will become much clearer or at least forecasts will be 
based on more years of data and experience and therefore are more likely to be accurate. 
If, as Method C projects, by the middle of this decade, it seems likely that only about 40 
to 50 percent of Kentucky’s schools will reach proficiency, I believe that policy makers 
will act as they did in 1998/99 to change the accountability system resulting in substantial 
increases in baseline scores for most schools. If such an adjustment occurs, then it is 
likely that as many as 800 or more schools (about two-thirds to three-quarters) will reach 
proficiency by 2014. 
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For political and symbolic reasons as well as educational reasons, most policy 
makers and education reformers will continue to insist that the goal is for all schools to 
reach proficiency or better, however even with substantial resources, interventions, and 
adjustments the goal of proficiency for all may not be realistic. Even with another major 
system change and scoring adjustment in the next decade, as many as 200-300 schools 
still may not reach the minimal goal, and without another major system change prior to 
2014, as many as 500 to 600 schools may not reach the goal. Given these possibilities, 
what will happen to the schools that do not meet the minimal standard by 2014? Will the 
state be prepared to take over any of these schools as is happening in Philadelphia, or will 
some schools simply be allowed to “fail?” Based on KDE reports and publicity, it 
appears that the Department of Education and many local school districts are working 
very hard to bring lagging schools to proficiency including scholastic audits and reviews, 
increased resources, and continued publicity and pressure to improve. Despite these 
efforts, achieving proficiency may not be possible in some schools and districts without 
radical changes in governance, leadership, and instructional programs. 
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APPENDIX A 

SUCCESSFUL SCHOOLS USING THREE FORECASTING METHODS * 

SCORE PROJECTED SCORE (2014) 

METHOD 



SCHOOL 


1993 


2001 


A 


B 


c 


ALLEN CO HS 


29.1 


63.7 


120 


108 


108 


ANDERSON CO MID 


33.4 


74 . 1 


140 


101 


108 


WESTERN EL 


41.6 


79 . 7 


142 


167 


108 


PAUL G BLAZER HS 


39.2 


75 . 7 


135 


140 


123 


HAGER EL 


40 . 6 


86.2 


160 


110 


121 


OAKVIEW EL 


36.6 


82.5 


157 


113 


135 


AUGUSTA EL 


21 . 7 


54.4 


108 


128 


100 


EASTERN EL 


40 . 9 


89 . 3 


168 


139 


134 


RED CROSS EL 


34 . 7 


80 . 3 


154 


107 


132 


A M YEALEY EL 


42.4 


79 . 8 


141 


107 


117 


BURLINGTON EL 


41.4 


75 . 6 


131 


103 


108 


CONNER MID 


38 . 8 


79 . 6 


146 


101 


117 


CONNER HS 


33 . 7 


77 . 7 


149 


111 


126 


HILLARD COLLINS EL 


35 . 8 


71.3 


129 


101 


114 


RYLE HIGH SCHOOL 


37 . 4 


77 . 7 


143 


114 


124 


NEW HAVEN EL 


38 . 3 


83 . 6 


157 


101 


129 


BOURBON CENTRAL EL 


34 . 2 


77 . 0 


147 


109 


122 


PARKER BENNETT EL 


20 . 0 


57.4 


118 


112 


104 


POTTER GRAY EL 


35 . 7 


89 . 8 


178 


165 


152 


BOWLING GREEN HS 


37 . 7 


73 . 3 


131 


114 


123 


T C CHERRY EL 


31.1 


85 .9 


175 


127 


156 


W R MCNEILL EL 


42 . 5 


92.8 


175 


127 


152 


BOYLE CO HS 


43 . 3 


74 .2 


124 


115 


108 


BRECKINRIDGE CO HS 


34.6 


71 . 7 


132 


107 


110 


BULLITT EAST HS 


38.4 


72 . 0 


127 


107 


112 


FIFTH DISTRICT EL 


31.4 


80 . 3 


160 


154 


143 


THIRD DISTRICT EL 


36 . 7 


80.1 


151 


113 


121 


CALDWELL CO HS 


27.2 


65 . 5 


128 


132 


117 


CALLOWAY CO HS 


38 . 5 


71.2 


124 


110 


115 


ALEXANDRIA EL 


38.1 


75 . 0 


135 


113 


115 


GRANTS LICK EL 


37 . 7 


87.2 


168 


105 


144 


JOLLY EL 


37.4 


79 . 3 


147 


116 


125 


CAMPBELL COUNTY HS 


34.6 


<T\ 

VO 


126 


109 


114 


CARLISLE CO HS 


45 . 0 


78 . 9 


134 


121 


125 


CARTER EL 


25.1 


70 . 0 


143 


142 


136 


EAST CARTER CO HS 


33 . 5 


61.5 


107 


123 


104 


OLIVE HILL EL 


31.7 


65.4 


120 


126 


110 


UPPER TYGART EL 


28 . 1 


77 . 0 


156 


138 


137 


PHELPS EL 


28 . 3 


73 . 3 


146 


114 


127 


BELMONT EL 


24 . 1 


64.6 


130 


119 


127 


HOLIDAY EL 


33 . 8 


76 . 8 


147 


142 


126 


LACY EL 


30.1 


65 . 5 


123 


117 


108 


MILLBROOKE EL 


31.6 


72.0 


138 


104 


115 


GEO ROGERS CLARK HS 


32 . 5 


68 . 5 


127 


117 


112 


TRAPP EL 


33 . 1 


95 . 2 


196 


153 


168 


BIG CREEK EL 


29 . 8 


74 . 6 


147 


115 


155 


CLAY CO HS 


25.2 


56 . 7 


108 


154 


120 


ONEIDA EL 


28 . 8 


82.5 


170 


116 


167 
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WILLIAM H NATCHER EL 34 . 
CRITTENDEN CO MID 41. 
DANVILLE HS 42 . 
TOLIVER EL 36. 
APOLLO HS 43. 
DAVIESS CO HS 38. 
HIGHLAND EL 43. 
MASONVILLE EL 47. 
PHILPOT EL 41. 
AUDUBON EL 35. 
TAMARACK EL 40. 
WEST LOUISVILLE EL 40. 
WHITESVILLE EL 34. 
BURNS EL 33. 
COUNTRY HEIGHTS EL 39. 
DAWSON SPRINGS HS 38. 
ELIZABETHTOWN HS 46. 
ARNETT EL 24 . 
HOWELL EL 32 . 
LLOYD HS 40. 
ATHENS EL 32. 
SCAPA AT BLUEGRASS 60. 
LAFAYETTE HS 43. 
LANSDOWNE EL 36. 
TRADITIONAL MAGNET 69. 
MAXWELL EL 31. 
STONEWALL EL 44 . 
TATES CREEK HS 39. 
CLAYS MILL EL 39. 
MEADOWTHORPE EL 47. 
P L DUNBAR HS 44. 
CHARLES CLARK EL 27. 
JOHNSON EL 48. 
MOYER EL 49. 
FRANKFORT HIGH SCHOOL 38. 
FRANKLIN CO HS 36. 
HEARN EL 34. 
WESTERN HILLS HS 35. 
GARRARD CO HS 31. 
PAINT LICK EL 30. 
E B TERRY EL 34 . 
GLASGOW HS 35. 
HAPPY VALLEY EL 32. 
SOUTH GREEN EL 33. 
CRITTENDEN MT ZION EL 27. 
DRY RIDGE EL 30. 
GRANT CO HS 31. 
MASON CORINTH EL 38. 
CUBA EL 41. 
FANCY FARM EL 42. 
SEDALIA EL 36. 
WINGO EL 33. 
H W WILKEY EL 35. 
CANEYVILLE EL 33. 
HANCOCK CO HS 37. 
HAWESVILLE EL 40. 
G C BURKHEAD EL 36. 



73 . 9 


138 


112 


107 


81.7 


147 


105 


118 


74 .4 


126 


146 


123 


70 . 0 


125 


119 


107 


77 . 5 


132 


131 


118 


77.6 


141 


133 


128 


91 . 5 


170 


151 


140 


106.9 


203 


164 


190 


89.0 


166 


107 


142 


84 . 6 


165 


163 


140 


84 . 8 


157 


132 


123 


90.4 


172 


171 


134 


84.4 


165 


114 


129 


80 . 5 


157 


125 


122 


89.8 


171 


126 


135 


75 . 3 


135 


108 


106 


83.2 


142 


124 


126 


79 . 0 


167 


179 


162 


78 . 0 


153 


127 


148 


77 . 9 


139 


154 


132 


81 . 8 


161 


137 


130 


98 . 0 


158 


115 


126 


82 . 1 


144 


128 


127 


89.1 


175 


112 


138 


97 . 9 


144 


102 


113 


92 .4 


191 


112 


133 


88 . 1 


160 


108 


130 


75.0 


133 


105 


110 


93 . 0 


181 


126 


134 


87 . 8 


153 


139 


138 


84 . 2 


149 


149 


136 


71.0 


142 


109 


111 


94.5 


170 


141 


127 


86.6 


147 


108 


102 


71.7 


126 


136 


105 


70 . 0 


125 


115 


110 


78 . 5 


150 


100 


117 


71 . 0 


129 


122 


113 


63.3 


115 


107 


100 


80 . 0 


160 


128 


146 


68 . 9 


124 


121 


114 


80 . 8 


155 


158 


143 


85.4 


172 


128 


140 


80 . 1 


156 


140 


135 


71.6 


143 


142 


127 


74 .7 


146 


145 


124 


69.5 


131 


113 


122 


80.2 


149 


115 


136 


82.5 


150 


116 


132 


86.5 


158 


110 


130 


82.2 


157 


135 


146 


83.2 


164 


127 


142 


75.2 


141 


106 


115 


77.1 


149 


121 


127 


73.3 


132 


122 


117 


84.2 


155 


112 


122 


72.8 


132 


111 


107 
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4 

7 

3 

8 

4 

1 

7 

5 

0 

3 

0 

5 

6 

9 

5 

8 

8 

1 

5 

9 

8 

9 

4 

8 

8 

0 

2 

1 

8 

5 

5 

2 

2 

2 

3 

4 

2 

6 

8 

8 

3 

4 

2 
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8 

9 

1 

1 

4 

0 

5 
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0 

3 

8 
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SONORA EL 37. 
UPTON EL 30. 
WESTERN EL 39. 
CENTRAL HARDIN HS 35. 
CAWOOD EL 27. 
NORTHS IDE EL 40. 
HAZARD HS 35. 
BEND GATE EL 34. 
CAIRO EL 31. 
HENDERSON CO HS 33. 
NIAGARA EL 33. 
NEW CASTLE EL 32. 
PRIDE AVENUE EL 28. 
WEST BROADWAY EL 42. 
EASTERN HS 39. 
FERN CREEK EL 37. 
FERN CREEK HS 22. 
GREATHOUSE SHRY EL 53. 
ROBERTA TULLY EL 45. 
AUDUBON TRAD EL 45. 
BUTLER TRAD HS 33. 
MALE HS 46. 
SCHAFFNER TRAD EL 41. 
WILDER EL 47. 
SENECA HS 37. 
INDIAN TRAIL EL 32. 
CRUMS LANE EL 24 . 
BOWEN EL 35. 
HITE EL 38. 
MINORS LANE EL 25. 
BALLARD HS 44. 
GUTERMUTH EL 25. 
LAYNE EL 31. 
LOWE EL 38. 
DUNN EL 44 . 
DUPONT MANUAL HS 51. 
BRANDEIS EL 36. 
WILMORE EL 39. 
BEECH GROVE EL 36. 
R C HINSDALE EL 41. 
FT WRIGHT EL 33. 
PINER EL 28. 
TAYLOR MILL EL 39. 
SCOTT HS 42. 
LARUE CO HS 34. 
BUSH EL 36. 
COLONY EL 35. 
SOUTH LAUREL HS 35. 
LAUREL EL 33. 
LIVINGSTON CENTRAL HS 29. 
ADAIRVILLE EL 28. 
AUBURN EL 37. 
MARY A GOETZ EL 36. 
LUDLOW HS 41. 
KINGSTON EL 27. 
KIT CARSON EL 41. 
WHITE HALL EL 38. 



72.1 


129 


123 


116 


76.7 


151 


122 


156 


64 . 9 


107 


109 


105 


68.6 


122 


112 


110 


65.6 


127 


102 


105 


86.0 


161 


174 


138 


68.0 


120 


129 


110 


71.3 


131 


107 


112 


70.6 


134 


120 


112 


69.7 


128 


110 


124 


72.9 


137 


110 


105 


72.4 


137 


117 


105 


74.5 


149 


112 


127 


79.6 


141 


105 


113 


72 . 9 


127 


108 


117 


80.0 


149 


113 


119 


63.4 


129 


119 


118 


93 . 8 


159 


102 


123 


84.7 


149 


105 


117 


91.9 


167 


133 


135 


79.7 


155 


139 


135 


90.5 


162 


153 


140 


88.6 


165 


114 


137 


82.1 


138 


108 


101 


70.5 


123 


133 


117 


68 . 5 


127 


111 


113 


65 . 5 


132 


105 


124 


75 . 1 


140 


109 


129 


85 . 3 


162 


133 


134 


81.5 


173 


136 


160 


81.3 


141 


146 


135 


59.8 


115 


104 


110 


64.5 


117 


113 


116 


89.3 


172 


112 


134 


79.9 


138 


113 


113 


94 . 5 


165 


134 


139 


80 . 8 


153 


115 


124 


84.8 


158 


127 


126 


73.2 


133 


116 


117 


86.2 


159 


116 


124 


75.7 


144 


122 


121 


71.6 


141 


115 


119 


79 . 0 


143 


112 


104 


72.2 


121 


102 


112 


67.6 


121 


113 


107 


78.0 


145 


106 


127 


69.3 


124 


104 


104 


72.9 


134 


152 


134 


75 . 9 


145 


117 


128 


66.5 


126 


127 


107 


80.2 


164 


135 


135 


77.4 


143 


101 


125 


85.0 


163 


126 


147 


85.7 


158 


113 


137 


66.0 


129 


105 


115 


81.5 


147 


108 


116 


77.1 


140 


109 


111 
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MILLARD HENSLEY EL 26. 
PRATER BORDERS EL 36. 
SALYER EL 29. 
WEST MARION EL 47. 
MASON CO HS 36. 
MAYFIELD HS 39. 
SPARKS EL 33. 
LONE OAK HS 30. 
STEARNS EL 32. 
CALHOUN EL 31. 
MCLEAN CO HS 29. 
EKRON EL 34. 
MEADE CO HS 37. 
MULDRAUGH EL 25. 
BOTTS EL 39. 
METCALFE CO HS 31. 
SUMMER SHADE EL 33. 
GAMALIEL EL 28. 
MONROE CO MIDDLE 33. 
JOE HARRISON CARTER EL 24. 
TOMPKINSVILLE EL 35. 
MONROE COUNTY HS 29. 
CAMARGO EL 30. 
MAPLETON EL 32. 
MONTICELLO HS 27. 
EZEL EL 37. 
WRIGLEY EL 39. 
CENTRAL CITY EL 38. 
GREENVILLE EL 35. 
DRAKESBORO CONSOL EL 38. 
HUGHES KIRK EL 31. 
LONGEST EL 32. 
MUHLENBERG NORTH HS 30. 
MUHLENBERG SOUTH HS 33. 
MURRAY HS 43. 
BOSTON EL 37. 
CHAPLIN EL 27. 
ELI H BROWN JR EL 28. 
MILDRED DEAN EL 29. 
NEWPORT HS 24. 
BEAVER DAM EL 36. 
OHIO CO HS 35. 
CENTERFIELD EL 39. 
GOSHEN ELEMENTARY 48. 
LAGRANGE EL 42. 
OLDHAM CO HS 42. 
OLDHAM CO MID 44. 
SOUTH OLDHAM HS 41. 
OWEN CO HS 36. 
CRAVENS ELEM 32. 
ESTES ELEM 36. 
FOUST ELEMENTARY 2 9 . 
NEWTON PARRISH EL 39. 
OWENSBORO HS 36. 
SEVEN HILLS EL 39. 
CLARK EL 45. 
PADUCAH TILGHMAN HS 31. 



81 . 8 


172 


173 


149 


68.9 


122 


125 


112 


73.7 


146 


100 


136 


91.4 


163 


108 


131 


69.1 


122 


100 


103 


69.9 


119 


103 


108 


66.8 


122 


105 


102 


74 . 6 


146 


150 


135 


68 . 9 


128 


124 


113 


86.1 


175 


132 


129 


65 . 1 


123 


125 


106 


81.2 


157 


125 


128 


71.0 


126 


114 


105 


73 . 0 


150 


139 


158 


73.8 


129 


107 


101 


63 . 9 


117 


114 


111 


73 .4 


138 


133 


106 


69.3 


136 


181 


164 


69.0 


127 


106 


106 


74.3 


155 


140 


147 


70 . 0 


127 


125 


117 


65 . 8 


125 


134 


127 


66 . 9 


125 


118 


113 


75.2 


144 


117 


111 


66.7 


131 


111 


112 


88 . 9 


172 


133 


153 


91.1 


176 


152 


161 


84 . 6 


159 


122 


123 


76 . 9 


144 


153 


124 


78 . 5 


144 


129 


132 


71.4 


136 


129 


119 


79.9 


157 


138 


128 


65 . 6 


123 


149 


112 


69.0 


127 


128 


117 


75.1 


126 


132 


116 


82.2 


154 


111 


129 


70.8 


141 


173 


137 


70.3 


138 


107 


107 


75 . 1 


14 9 


116 


118 


59.9 


117 


120 


107 


70.7 


126 


112 


107 


67.2 


119 


119 


105 


97 . 6 


192 


143 


151 


106.2 


201 


152 


166 


75.1 


129 


118 


121 


79 . 9 


141 


102 


110 


89.2 


161 


117 


127 


83.6 


152 


137 


136 


70 . 0 


125 


116 


105 


75.0 


144 


123 


120 


75 . 0 


138 


111 


113 


75 . 0 


14 9 


133 


126 


75 . 0 


133 


102 


108 


69.1 


123 


109 


109 


75 . 0 


133 


101 


107 


79.5 


135 


109 


118 


71.1 


135 


126 


120 
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PAINTSVILLE EL 


44 . 2 


82.2 


144 


104 


113 


DENNIS C WOOTON EL 


28.6 


74.3 


149 


108 


123 


BELFRY HS 


27 . 9 


63.7 


122 


115 


118 


FEDS CREEK HS 


34.7 


62 . 5 


108 


109 


101 


GEORGE F JOHNSON EL 


40.6 


92.6 


177 


129 


156 


JOHNS CREEK EL 


30.6 


73.0 


142 


123 


121 


SHELBY VALLEY HS 


31.4 


63.5 


116 


120 


111 


PINEVILLE HS 


29.3 


68.7 


133 


112 


118 


BOWEN EL 


29.7 


97.7 


208 


136 


156 


CLAY CITY EL 


32.5 


78 . 0 


152 


125 


126 


PULASKI EL 


33.7 


75.4 


143 


106 


133 


WOODSTOCK EL 


26 . 5 


72 . 8 


148 


132 


129 


SOUTHERN EL 


33.4 


63.3 


112 


104 


106 


DEMING HS 


28 . 9 


62.3 


117 


105 


106 


ROCKCASTLE CO HS 


31.3 


65.5 


121 


117 


107 


CLEARFIELD EL 


38.5 


77.3 


140 


104 


112 


RUSSELL SPRINGS EL 


35 . 5 


77.2 


145 


106 


126 


RUSSELL CO HS 


35.2 


68.3 


122 


117 


112 


SALEM EL 


36 . 1 


84 . 6 


163 


109 


127 


RUSSELL HS 


39.4 


72 . 5 


126 


114 


109 


RUSSELLVILLE HS 


43 . 1 


70.7 


116 


117 


109 


SCOTT CO SR HS 


34 . 1 


67.0 


120 


109 


104 


MANNSVILLE EL 


28 . 0 


74 . 0 


149 


107 


120 


NORTH TODD EL 


31.7 


69 . 7 


131 


100 


112 


SOUTH TODD EL 


27 .7 


70 . 6 


140 


109 


124 


MILTON EL 


34 .0 


75 . 3 


142 


127 


123 


UNION CO HS 


37 . 6 


68 . 3 


118 


123 


103 


WALTON VERONA EL 


36 . 1 


88.4 


173 


137 


152 


WALTON VERONA HS 


39.2 


85.3 


160 


138 


142 


ALVATON EL 


36.6 


79.0 


148 


108 


120 


OAKLAND EL 


33 . 0 


76 . 1 


146 


105 


150 


WARREN CENTRAL HS 


38 . 0 


65 . 9 


111 


119 


103 


GREENWOOD H S 


41.6 


80.5 


144 


112 


118 


WEBSTER CO HS 


29.3 


68 . 8 


133 


133 


122 


POPLAR CREEK EL 


25.4 


70.7 


144 


105 


123 


WILLIAMSBURG EL 


36.9 


73.4 


133 


126 


104 


WILLIAMS TOWN EL 


36 . 0 


79 . 9 


151 


104 


130 


ROGERS EL 


33.3 


79 . 3 


154 


132 


122 


SOUTHS IDE EL 


39.8 


82.4 


152 


126 


121 


HUNTERTOWN EL 


34.7 


81.3 


157 


131 


122 


MODEL LAB EL 


48.4 


82 . 8 


139 


103 


109 


MODEL LAB HS 


52 . 6 


84.3 


136 


117 


118 



* These are the 269 schools that operated in 1993, continued through 2001, and are projected to reach the 
minimum goal of 100 points using all three methods. These projections use the 1999 and 2000 
accountability scores as adjusted in 2001. 
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APPENDIX B 

PROJECTED SUCCESSFUL URBAN SCHOOLS 



SCORES 



SCHOOL 


1993 


2001 


2014 


ATHENS EL 


32.9 


81.8 


129.9 


SCAPA AT BLUEGRASS 


60.8 


98.0 


126 . 0 


CASSIDY EL 


50.5 


88.9 


124.2 


JULIA R EWAN EL 


38.0 


73.8 


100.4 


LAFAYETTE HS 


43.9 


82.1 


127 . 0 


LANSDOWNE EL 


36.4 


89.1 


137 . 6 


TRADITIONAL MAGNET 


69.8 


97.9 


112 . 9 


LINLEE EL 


26.4 


69.0 


125.1 


MAXWELL EL 


31.8 


92.4 


133 .4 


SQUIRES EL 


39.6 


74 .4 


102 . 3 


STONEWALL EL 


44 . 0 


88.1 


130.4 


RUSSELL EL 


25 . 8 


64.8 


118 . 7 


TATES CREEK HS 


39.2 


75 . 0 


110 . 5 


CLAYS MILL EL 


39.1 


93 . 0 


134 . 0 


GLENDOVER EL 


49.6 


83.4 


104 . 9 


MEADOWTHORPE EL 


47 . 8 


87.8 


137.6 


BRYAN STATION HS 


31 . 3 


62.2 


101 . 0 


MARY TODD EL 


28 . 3 


57.7 


102.1 


P L DUNBAR HS 


44.5 


84.2 


136.2 


EASTERN HS 


39.6 


72 . 9 


117.1 


FERN CREEK EL 


37 . 8 


80 . 0 


119.2 


FERN CREEK HS 


22 . 9 


63.4 


117.6 


GREATHSE SHRY TRAD EL 


53 . 6 


93 . 8 


123 . 1 


ROBERTA TULLY EL 


45 . 4 


84 . 7 


116.6 


BARRET MID 


49.3 


84.4 


106.5 


AUDUBON TRAD EL 


45 . 8 


91.9 


135 . 0 


BUTLER TRAD HS 


33 . 5 


79 . 7 


134.5 


MALE HS 


46.3 


90.5 


140.1 


GILMORE LANE EL 


30.4 


69.9 


118 . 7 


GOLDSMITH LANE EL 


36.6 


66 . 0 


101.5 


SCHAFFNER TRAD EL 


41.6 


88.6 


136 . 9 


ST MATTHEWS EL 


41.8 


80.8 


111.3 


WILDER EL 


47 . 9 


82 . 1 


100 . 9 


SENECA HS 


37 . 9 


70.5 


117 . 5 


INDIAN TRAIL EL 


32.6 


68 . 5 


114 . 0 


Z TAYLOR EL 


42.7 


76.2 


105 . 5 


KERRICK EL 


43.1 


76 . 7 


113 . 5 


CRUMS LANE EL 


24 .7 


65.5 


123.6 


BOWEN EL 


35.2 


75.1 


129.1 


HITE EL 


38 . 0 


85.3 


133 . 8 


NORTON EL 


49.4 


81.1 


103.2 


MINORS LANE EL 


25 . 0 


81.5 


160.2 


BALLARD HS 


44.5 


81.3 


135.3 


LUHR EL 


34.7 


66.3 


102.1 


GUTERMUTH EL 


25.7 


59.8 


110.1 


LAYNE EL 


31.9 


64.5 


116.3 


LOWE EL 


38.4 


89.3 


134 . 1 


MILL CREEK EL 


26.0 


63.9 


110 . 7 


DUNN EL 


44.2 


79.9 


113.1 


JEFFERSONTOWN EL 


35.3 


71.8 


124.2 


DUPONT MANUAL HS 


51.2 


94.5 


138.7 
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ENGELHARD EL 


24.9 


78.0 


146 . 9 


BRANDEIS EL 


36.6 


80.8 


123 . 5 


BROWN EL 


47.6 


77.5 


101 . 5 


BROWN HS 


45.2 


77.5 


105 . 9 


FOSTER EL 


24 .7 


65 . 8 


111.1 


MCFERRAN EL 


35.2 


71 . 8 


109.5 


L T JOHNSON TRAD MS 


34 .2 


73.5 


106 . 0 


CARTER TRAD EL 


40.7 


88.6 


130 . 6 



* These are the 59 urban schools that operated in 1993, continued through 2001, and are projected to reach 
the minimum score of 100 by 2014 using Method C. These projections use 1999 and 2000 accountability 
scores as adjusted in 2001. 
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APPENDIX C 

TIMELINE FOR KENTUCKY’S ACCOUNTABILITY POLICIES 

AND SYSTEMS 



MARCH 1997 

A Task Force begins an examination of the existing KIRIS accountability system including 
contentious issues such as use of a national standardized or norm-referenced test along with or 
instead of the “custom-made” tests based on Kentucky specific content and standards, use of 
rewards as salary bonuses to staff and types of school sanctions, and whether each school should 
have a fixed improvement goal. 

DECEMBER 1997 

The Task Force issues a report for the 1998 session of the General Assembly that recommends 
use of some type of national test, increased emphasis on basic academic skills, simplifying school 
achievement goals, cash rewards not be used as bonuses for teachers, and development of a 
school report card. (Harp, Lonnie, December 10, 1997. Task Force Urges Changes in KERA Test. 
Louisville Courier-Journal) 

JANUARY 1998 

A report from a group of national testing experts commissioned by the Legislative Research 
Commission is critical of the KIRIS accountability system. Based on an audit of the scoring 
system, the panel concludes that the scoring index inflates actual scores and scores prior to 1996 
be used for information only. The panel recommends that a process be initiated to reestablish a 
baseline year and reset standards as soon as possible. (Blackford, Linda. January 11, 1998. Toss 
Out Early Scores on KIRIS Tests and Start Over, Experts Say. Lexington Herald-Leader) 

MAY 1998 

The General Assembly approves HB 53 requiring that the new Commonwealth Accountability 
Testing System (CATS) include a norm-referenced test that matches Kentucky’s core content and 
provides valid and reliable results for the individual student, open-response and multiple -choice 
items, writing portfolios but with reduced time for students and teacher involvement, extensive 
involvement of teachers in designing and scoring the new test, and other components. The 
legislation also requires the establishment of a school report card, reconfigures the consequences 
section of accountability, and requires the State Board to school to establish a new formula for 
accountability and improvement. 

MAY 1998 

A group of legislators, educators and citizens (School Curriculum, Assessment, and 
Accountability Council) is appointed to advise the state School Board on further development of 
a new state testing system. The group is to have a design for a new test recommended to the state 
board in time to meet a June 15 deadline for seeking bids on the test. Issues to be addressed 
include whether the test should be designed to provide reliable scores for individuals students, 
schools, or both; the mix of multiple -choice and essay questions; and how much of the test is to 
be customized for Kentucky or consist of national standardized test items. (Harp, Lonnie, May 
13, 1998. Group tackles State Testing Redesign Issues. Louisville Courier-Journal) 

JUNE 1998 

Board approves RFP for proposals to build a new accountability test. 
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SEPTEMBER 1998 

CTB/McGraw Hill awarded four-year $30 million contract by Board of Education to design and 
score CATS as the replacement for KIRIS accountability system. 

OCTOBER 1998 

The Commissioner of Education cites improvements in the CATS system. The development 
process was collaborative (teachers, parents, administrators, testing experts, etc.); the test is more 
reliable and valid; testing will take less time; results will be reported sooner; results can be used 
to measure the progress of individual students over time; and the contractor has resources and a 
proven track record. (Commissioner’s Comments, Kentucky Teacher, October 1998) 

DECEMBER 1998 

The state school board proposes a new system to evaluate and classify schools. A straight line or 
“line of expected growth” is plotted for each school showing how it must progress from 2000 to 
reach the goal of 100 points by 2014. If two-year interim goals on the straight line are met or 
exceeded, the school receives cash rewards (to receive rewards, schools also must meet additional 
standards for reducing drop-out rates and decreasing the number of students scoring in the novice 
category). Schools with lower scores will have steeper lines than schools with higher schools, but 
all schools will have a safety zone under the line that allows them to slip a few points without 
sanctions. Every two years, a school will be classified as meeting its goal, progressing, or needing 
assistance. Some schools in the progressing range may receive partial awards, and schools falling 
below the assistance line may be eligible for a “scholastic audit” to determine the type of 
assistance needed. 

FEBRUARY 1999 

Based on recommendations from The National Technical Advisory Panel on Assessment and 
Accountability and WestEd, the assessment subcontractor for the Kentucky Core Content Tests 
that the core content be reviewed prior to the development of new questions, committees begin 
benchmarking (aligning to national standards) in five content areas 

APRIL 1999 

The first CATS test is taken 
AUGUST 2001 

Recalculated scores for 1999 and 200 are released in the form of “growth charts” for each 
individual school. A model or example of the growth chart with terms used can be viewed at 
http://www.kde.state.kv.us/comm/commrel/cats/long term.asp 



New performance standards are found at 
http://www.kde.state.kv.us/comm/pubinfo/standards/ 



The various components of the CATS testing system and the grades where each is 
administered are described in the table below. 



CATS STUDENT TESTING MATRIX 






STUDENT GRADE 


TEST 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


Norm Referenced Test * 


X 






X 






X 








Reading 




X 






X 






X 






Math 






X 






X 






X 




Science 




X 






X 








X 




Social Studies 






X 






X 






X 




Writing 




X 






X 










X 


Arts & Humanities 






X 






X 






iX| 




Practical Living & Vocat 






X 






X 




X 






* The norm referenced test is CTBS/5 
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